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DETAILED ACTION 

Continued Examination Under 37 CFR 1. 1 14 

1 . A request for continued examination under 37 CFR 1.114, including the fee set 
forth in 37 CFR 1 .1 7(e), was filed in this application after final rejection. Since this 
application is eligible for continued examination under 37 CFR 1.114, and the fee set 
forth in 37 CFR 1 .1 7(e) has been timely paid, the finality of the previous Office action 
has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 
08/02/2010 has been entered. 

Response to Arguments 

2. Applicant's arguments, filed 08/02/201 0, with respect to the 35 U.S.C. 1 02 and 
103 rejections of claims 1-1 1 and 13-24 have been fully considered but they are not 
persuasive. 

3. On page 23 of Applicant's Arguments, Applicant argues that Subramaniyan fails 
to disclose: 

a) determining, using one or more computer processing systems, a motion 
estimation representative of the global motion between the particular frame and a first 
preceding or succeeding frame of the inter-frame encoded video sequence, on the basis 
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of motion vectors directly between the particular frame and the first preceding or 
succeeding frame; 

b) determining, using the one or more computer processing systems, one or 
more further motion estimations representative of the global motion between the 
particular frame and the first preceding or succeeding frame at least partially on the 
basis of motion vectors between the particular frame and one or more preceding or 
succeeding other frames; and 

c) selecting, using the one or more computer processing systems, one of the 
motion estimations which meets at least one predetermined criterion as being 
representative of the global motion of the particular frame; 

Examiner respectfully disagrees. 

As stated in [0030] of Subramaniyan, the "global motion vector is estimated by 
the motion estimation circuit 1 10 as the average of all final motion vectors of the 
previous frame". Thus Subramaniyan is able to determine a motion estimation 
representative of the global motion between the particular frame and another frame on 
the basis of an average of motion vectors therebetween, which satisfies claim limitation 
(a). This is reaffirmed in [0035] of Subramaniyan, which states that "the motion 
estimation circuit 1 10 can determine the global motion vector by using an average of all 
final motion vectors in a previous frame". 
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As stated in [0034] of Subramaniyan, "further motion vector predictors can be 
determined based on the result of motion estimation done for the same macroblock, but 
on a different previously coded frame". Thus Subramaniyan discloses further motion 
vector predictors based on a previous frame that is different from the previous frame 
initially used. Using these further motion vectors predictors in light of how global motion 
vectors in Subramaniyan are determined, Subramaniyan is capable of determining a 
further motion estimation representative of the global motion between the particular 
frame and another frame at least partially on the basis of an average of motion vectors 
between the particular frame and one or more different other frames, which satisfies 
limitation (b). This is reaffirmed in [0026] of Subramaniyan, which states that "motion 
estimation is computed for blocks of image data from a current image frame using one 
or more previously processed image frames". 

As stated in [0030] of Subramaniyan, the "global motion vector is estimated by 
the motion estimation circuit 1 10 as the average of all final motion vectors of the 
previous frame for which a difference metric was below a certain threshold THRESH1 ". 
Since the final motion vectors are subject to a difference metric and the final motion 
vectors are used to form the global motion vector, the global motion vector is subject to 
a predetermined criterion. By determining global motion in such a manner, global 
motion is contemporaneously selected upon its determination. Thus Subramaniyan is 
capable of selecting one of the motion estimations which meets at least one 
predetermined criterion as being representative of the global motion of the frame, which 
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satisfies limitation (c). This is reaffirmed in [0035] of Subramaniyan, which states that 
"the motion estimation circuit 1 10 can determine the global motion vector by calculating 
a difference metric for each of final motion vectors in a previous frame, comparing the 
difference metric for each of the final motion vectors in the previous frame with a 
predetermined threshold, and determining the global motion vector based on the each 
of the final motion vectors in a previous frame with a difference metric that is below the 
threshold". 

4. In the middle of page 24 of Applicant's Arguments, Applicant states that "[i]n part 
(c) of claim 1 , the separate estimates are combined to arrive at single estimate of global 
motion between the different frames." On the last line of page 25 of Applicant's 
Arguments, Applicant similarly states that "the single vector characterizing global motion 
is derived from a combination of two independently calculated global motion values." 
Examiner notes that while this may be true of Applicant's invention as disclosed in 
Applicant's specification, this type of "combination" does not appear in Applicant's 
independent claims. 

5. On page 26 of Applicant's Arguments, Applicant argues that "Subramaniyan uses 
predictor motion vectors to match pixel blocks (macroblocks) in preparation for encoding 
an image" and that "Subramaniyan is directed to a video encoder system", while "[i]n the 
invention of claim 1 , motion estimation is applied to an already encoded image in order 
to allow the plurality of vectors to be replaced by a single vector, e.g. for the purposes of 
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image registration". In support of this argument, Applicant states that "Subramaniyan 
thus describes processing raw pixel data in an unencoded frame with the obvious intent 
of encoding the frame in the video encoder. As such, Subramaniyan could be seen as a 
precursor stage to the method of the invention of claim 1 , in that Subramaniyan 
describes generating an encoded signal of the type that is processed by the present 
invention. However, since Subramaniyan could be seen as a precursor stage to the 
method of the invention of claim 1 , Subramaniyan clearly fails to disclose features (a)- 
(c) of claim 1 . The invention of claim 1 deals with a video signal that is already encoded, 
as stated by the claim 1 ." Examiner respectfully disagrees with such a narrow 
interpretation of Subramaniyan. 

Examiner agrees with Applicant that "Subramaniyan uses predictor motion 
vectors to match pixel blocks (macroblocks) in preparation for encoding an image" and 
that "the invention of claim 1 , motion estimation is applied to an already encoded image 
in order to allow the plurality of vectors to be replaced by a single vector, e.g. for the 
purposes of image registration". However, the fact that Subramaniyan can use global 
motion as a motion vector predictor in preparation for coding future frames does not by 
itself negate the fact that, in Subramaniyan, global motion is calculated in a manner 
consistent with Applicant's claim language. This simply indicates that, in Subramaniyan, 
global motion can be used for motion estimation of subsequent frames. There is 
nothing in Applicant's claim language which prevents global motion from being used as 
a motion vector predictor in the motion estimation of subsequent frames. The global 
motion in Subramaniyan can be implemented for a variety of uses and still disclose 
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Applicant's claim limitations (a)-(c) so long as the Subramaniyan global motion is 
calculated in a manner consistent with Applicant's claim language. 

Applicant argues that since Subramaniyan processes raw pixel data to generate 
an encoded signal, Subramaniyan can only be viewed as a precursor to Applicant's 
claims. Applicant continues this logic and argues that because Subramaniyan can only 
be viewed as a precursor to Applicant's claims, Subramaniyan cannot disclose claim 
limitations (a)-(c) which is for an already encoded signal. 

From a general standpoint, Applicant is correct that Subramaniyan processes a 
raw pixel data input to generate an encoded signal output. However, in between this 
raw pixel data being input and an encoded signal being output, Subramaniyan discloses 
using an inter-frame encoded signal to generate global motion. As stated in [0035] of 
Subramaniyan, "the motion estimation circuit 110 can determine the global motion 
vector by using an average of all final motion vectors in a previous frame". By using 
motion vectors to calculate global motion, raw pixel data is not used to calculate global 
motion. Raw pixel data does not contain motion vectors. The use of motion vectors 
between frames indicates that these frames have been previously inter-frame encoded. 
This is because inter-frame encoded data by definition includes at least some indication 
of similarities and differences between frames, such as motion vectors and/or residual 
data. It is noted that such a definition of inter-frame encoded data is consistent with 
Applicant's specification, which defines inter-frame encoding to be the production of 
"motion vectors indicative of the general motion of a number of macroblocks which 
make up a frame with respect to a preceding or succeeding frame", as stated in page 
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29, line 3-7, of Applicant's specification. Thus, by using motion vectors to calculate 
global motion, as in Subramaniyan, previously inter-frame encoded video data is used 
to calculate global motion. 

6. In the top of page 27 of Applicant's Arguments, Applicant states that "[a]s 
indicated at paragraph [0030] [of Subramaniyan], one option for the predictor motion 
vectors used in Subramaniyan is an estimate of global motion for a previous frame (i.e., 
not for the current frame, as in the invention of claim 1 )". Examiner notes that while this 
may be true of Applicant's invention as disclosed in Applicant's specification, use of a 
"current frame" does not appear in Applicant's independent claims. 

7. In the middle of page 27 of Applicant's Arguments, Applicant argues that "[a]s 
with the other predictor motion vectors, the estimated global motion vector [of 
Subramaniyan] is not used to represent global motion of the current frame, but to 
indicate where in the current frame to start a search for the best individual macroblock 
motion vectors with which to encode the current frame". Examiner respectfully 
disagrees with such a narrow in interpretation of Subramaniyan, as shown above. 
Subsequent use of global motion in Subramaniyan does not negate creation of global 
motion in a certain manner, as shown above. Furthermore, the global motion of 
Subramaniyan is calculated as representative of a frame based on motion vectors 
between that frame and another frame, as shown above. 
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Claim Objections 

8. Claims 1 4 and 1 9 are objected to because of the following informalities: 
Limitations (i) recite "an preceding or succeeding frame", which is grammatically 
incorrect. Appropriate correction is required. 

Claim Rejections - 35 USC §112 

9. The following is a quotation of the second paragraph of 35 U.S.C. 112: 

The specification shall conclude with one or more claims particularly pointing out and distinctly 
claiming the subject matter which the applicant regards as his invention. 

1 0. Claims 1 , 13-14 and 1 9 are rejected under 35 U.S.C. 1 1 2, second paragraph, as 
being indefinite for failing to particularly point out and distinctly claim the subject matter 
which applicant regards as the invention. 

1 1 . Claims 1 and 1 3 recite a "first preceding or succeeding frame" three times in 
each claim. It is unclear if the word "first" applies only to a "preceding" frame or if the 
word "first" applies to both a "preceding" frame as well as a "succeeding" frame. 
Examination was conducted as if the word "first" applies to both a "preceding" frame as 
well as a "succeeding" frame. 

1 2. Claims 1,13-14 and 1 9 recite "the particular frame" and a "preceding or 
succeeding frame of the inter-frame encoded video sequence". It is unclear if only the 
"preceding or succeeding frame" are "of the inter-frame encoded video sequence" or if 
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both "the particular frame" as well as the "preceding or succeeding frame" are "of the 
inter-frame encoded video sequence". Examination was conducted as if both "the 
particular frame" as well as "a first preceding or succeeding frame" are "of the inter- 
frame encoded video sequence" 

1 3. Claims 1 and 1 3 recite the limitation "the image" in the last paragraph of each 
claim. There is insufficient antecedent basis for this limitation in the claims. 

Claim Rejections - 35 USC § 102 

1 4. The following is a quotation of the appropriate paragraphs of 35 U.S.C. 1 02 that 
form the basis for the rejections under this section made in this Office action: 

A person shall be entitled to a patent unless - 

(e) the invention was described in (1) an application for patent, published under section 122(b), by 
another filed in the United States before the invention by the applicant for patent or (2) a patent 
granted on an application for patent by another filed in the United States before the invention by the 
applicant for patent, except that an international application filed under the treaty defined in section 
351 (a) shall have the effects for purposes of this subsection of an application filed in the United States 
only if the international application designated the United States and was published under Article 21 (2) 
of such treaty in the English language. 

15. Claims 14-17 are rejected under 35 U.S.C. 102(e) as being anticipated by 
Subramaniyan et al. (US Patent Application Publication # 2004/0028134 A1). 

16. Regarding claim 14, 
Subramaniyan discloses: 
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A system for global motion estimation between frames of a motion-compensated inter- 
frame encoded video sequence, the system comprising: 
video processing means arranged in use to: 

i) determine a motion estimation representative of the global motion 
directly between the particular frame and preceding or succeeding frame of the 
inter-frame encoded video sequence on the basis of motion vectors 
therebetween (As stated in [0030] of Subramaniyan, the "global motion vector is 
estimated by the motion estimation circuit 1 10 as the average of all final motion 
vectors of the previous frame". Thus Subramaniyan is able to determine a 
motion estimation representative of the global motion between the particular 
frame and another frame on the basis of an average of motion vectors 
therebetween, which satisfies claim limitation (a). This is reaffirmed in [0035] of 
Subramaniyan, which states that "the motion estimation circuit 110 can 
determine the global motion vector by using an average of all final motion vectors 
in a previous frame".); 

ii) determine one or more further motion estimations representative of the 
global motion between the particular frame and the preceding or succeeding 
frame at least partially on the basis of motion vectors between the particular 
frame and one or more preceding or succeeding other frames (As stated in 
[0034] of Subramaniyan, "further motion vector predictors can be determined 
based on the result of motion estimation done for the same macroblock, but on a 
different previously coded frame". Thus Subramaniyan discloses further motion 
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vector predictors based on a previous frame that is different from the previous 
frame initially used. Using these further motion vectors predictors in light of how 
global motion vectors in Subramaniyan are determined, Subramaniyan is 
capable of determining a further motion estimation representative of the global 
motion between the particular frame and another frame at least partially on the 
basis of an average of motion vectors between the particular frame and one or 
more different other frames, which satisfies limitation (b). This is reaffirmed in 
[0026] of Subramaniyan, which states that "motion estimation is computed for 
blocks of image data from a current image frame using one or more previously 
processed image frames".); and 

a motion estimation selector means arranged in use to select one of the motion 
estimations which meets at least one predetermined criterion as being representative of 
the global motion of the frame (As stated in [0030] of Subramaniyan, the "global motion 
vector is estimated by the motion estimation circuit 1 10 as the average of all final motion 
vectors of the previous frame for which a difference metric was below a certain 
threshold THRESH1 ". Since the final motion vectors are subject to a difference metric 
and the final motion vectors are used to form the global motion vector, the global motion 
vector is subject to a predetermined criterion. By determining global motion in such a 
manner, global motion is contemporaneously selected upon its determination. Thus 
Subramaniyan is capable of selecting one of the motion estimations which meets at 
least one predetermined criterion as being representative of the global motion of the 
frame, which satisfies limitation (c). This is reaffirmed in [0035] of Subramaniyan, which 
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states that "the motion estimation circuit 110 can determine the global motion vector by 
calculating a difference metric for each of final motion vectors in a previous frame, 
comparing the difference metric for each of the final motion vectors in the previous 
frame with a predetermined threshold, and determining the global motion vector based 
on the each of the final motion vectors in a previous frame with a difference metric that 
is below the threshold".). 

17. Regarding claim 15, 
Subramaniyan discloses: 

A system according to claim 14, wherein the video processing means is further 
arranged in use to: 

determine one or more motion estimations representative of the global motion of 
the frame with respect to a one or more respective preceding or succeeding other 
frames (Subramaniyan: [0035]; [0026], motion estimation is computed for blocks of 
image data from a current image frame using one or more previously processed image 
frames ) 

determine one or motion estimations respectively representative of the global 
motion of the one or more other frames with respect to the preceding or succeeding 
frame (Subramaniyan: [0035]; the motion estimation circuit 1 10 can determine the 
global motion vector by using an average of all final motion vectors in a previous frame ); 
and 
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accumulate the respective motion estimations to give one or more respective 
overall motion estimations each substantially representative of the global motion of the 
frame with respect to the preceding or succeeding frame (Subramaniyan: [0035], the 
motion estimation circuit 1 1 0 can determine the global motion vector by using an 
average of all final motion vectors in a previous frame). 

18. Regarding claim 16, 
Subramaniyan discloses: 

A system according to claim 14, wherein the motion estimation selector means 
further comprises motion estimation testing means for testing the motion estimations in 
turn (Subramaniyan: [0035], comparing the difference metric for each of the final motion 
vectors in the previous frame with a predetermined threshold; [0049], This process 
continues until one of 3 conditions are met: [0050], CONDITION 1 : The MSAD is below 
a threshold THRESH4 , given by: THRESH4=A*Q+B), and providing an indication as to 
whether the test is passed; and output means arranged in use to output a motion 
estimation as being representative of the global motion of the frame in dependence on 
the receipt of an indication that the test was passed from the motion estimation testing 
means, wherein the motion estimation testing means is further arranged to apply the 
test in turn to motion estimations once they have been determined, and if the test is 
passed then no further motion estimations are determined (Subramaniyan: [0035], 
determining the global motion vector based on the each of the final motion vectors in a 
previous frame with a difference metric that is below the threshold; [0049], This process 
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continues until one of 3 conditions are met: [0050], CONDITION 1 : The MSAD is below 
a threshold THRESH4 , given by: THRESH4=A*Q+B), and if the test is passed then no 
further motion estimations are determined (Subramaniyan: [0049], This process 
continues until one of 3 conditions are met : [0050], CONDITION 1 : The MSAD is below 
a threshold THRESH4, given by: THRESH4=A*Q+B). 

19. Regarding claim 17, 
Subramaniyan discloses: 

A system according to claim 16, wherein the test comprises comparing the 
motion estimation with a threshold value (Subramaniyan: [0049]-[0050], THRESH4), 
wherein the test is passed if the parameters of the motion estimation do not exceed the 
threshold value (Subramaniyan: [0049]-[0050], [0054] The best MV at the end of the 
second stage is chosen as the best MV for the macroblock; the MV having a MSAD 
below THRESH4 is chosen as the best MV). 

Claim Rejections - 35 USC § 103 

20. The following is a quotation of 35 U.S.C. 1 03(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 1 02 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 
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21 . Claims 1 -4, 6-9, 1 3 and 1 9-22 are rejected under 35 U.S.C. 1 03(a) as being 
unpatentable over Subramaniyan et al. (US Patent Application Publication # 
2004/00281 34 A1 ) in view of Jinzenji et al. (US Patent # 6,977,664 B1 ). 

22. Regarding claim 1 , 
Subramaniyan teaches: 

A computer-implemented method of global motion estimation between frames of a 
motion-compensated inter-frame encoded video sequence, the method comprising, for 
a particular frame: 

a) determining, using one or more computer processing systems, a motion 
estimation representative of the global motion between the particular frame and a first 
preceding or succeeding frame of the inter-frame encoded video sequence, on the basis 
of motion vectors directly between the particular frame and the first preceding or 
succeeding frame (As stated in [0030] of Subramaniyan, the "global motion vector is 
estimated by the motion estimation circuit 1 10 as the average of all final motion vectors 
of the previous frame". Thus Subramaniyan is able to determine a motion estimation 
representative of the global motion between the particular frame and another frame on 
the basis of an average of motion vectors therebetween, which satisfies claim limitation 
(a). This is reaffirmed in [0035] of Subramaniyan, which states that "the motion 
estimation circuit 1 10 can determine the global motion vector by using an average of all 
final motion vectors in a previous frame".); 
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b) determining, using the one or more computer processing systems, one or 
more further motion estimations representative of the global motion between the 
particular frame and the first preceding or succeeding frame at least partially on the 
basis of motion vectors between the particular frame and one or more preceding or 
succeeding other frames (As stated in [0034] of Subramaniyan, "further motion vector 
predictors can be determined based on the result of motion estimation done for the 
same macroblock, but on a different previously coded frame". Thus Subramaniyan 
discloses further motion vector predictors based on a previous frame that is different 
from the previous frame initially used. Using these further motion vectors predictors in 
light of how global motion vectors in Subramaniyan are determined, Subramaniyan is 
capable of determining a further motion estimation representative of the global motion 
between the particular frame and another frame at least partially on the basis of an 
average of motion vectors between the particular frame and one or more different other 
frames, which satisfies limitation (b). This is reaffirmed in [0026] of Subramaniyan, 
which states that "motion estimation is computed for blocks of image data from a current 
image frame using one or more previously processed image frames".); and 

c) selecting, using the one or more computer processing systems, one of the 
motion estimations which meets at least one predetermined criterion as being 
representative of the global motion of the particular frame (As stated in [0030] of 
Subramaniyan, the "global motion vector is estimated by the motion estimation circuit 
1 1 0 as the average of all final motion vectors of the previous frame for which a 
difference metric was below a certain threshold THRESH1 ". Since the final motion 
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vectors are subject to a difference metric and the final motion vectors are used to form 
the global motion vector, the global motion vector is subject to a predetermined 
criterion. By determining global motion in such a manner, global motion is 
contemporaneously selected upon its determination. Thus Subramaniyan is capable of 
selecting one of the motion estimations which meets at least one predetermined 
criterion as being representative of the global motion of the frame, which satisfies 
limitation (c). This is reaffirmed in [0035] of Subramaniyan, which states that "the 
motion estimation circuit 1 10 can determine the global motion vector by calculating a 
difference metric for each of final motion vectors in a previous frame, comparing the 
difference metric for each of the final motion vectors in the previous frame with a 
predetermined threshold, and determining the global motion vector based on the each 
of the final motion vectors in a previous frame with a difference metric that is below the 
threshold".); 

d) storing or outputting said selected motion estimation (Subramaniyan: [0030]; 
[0076]; Fig 6: item 620: output of global motion estimation) 

Subramaniyan fails to teach: 

d) storing or outputting said selected motion estimation for use in registering the 
image of said particular frame with the image of said first preceding or succeeding 
frame. 



Jinzenji teaches: 
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d) storing or outputting said selected motion estimation for use in registering the 
image of said particular frame with the image of said first preceding or succeeding 
frame (Jinzenji: Fig 3: S1 ; Fig 4: 1 ; col 8, line 29 - col 9, line 16: output of global motion 
for use in frame mapping to a reference coordinate system). 

At the time of invention, it would have been obvious to a person having ordinary 
skill in the art to combine the teachings of Jinzenji with Subramaniyan. Outputting the 
global motion for use in image registration, as in Jinzenji, would allow for creation of a 
panoramic image based on the global motion of Subramaniyan, thereby allowing a user 
to implement an established use for global motion. 

23. Regarding claim 2, Subramaniyan in view of Jinzenji teaches: 

A method according to claim 1 , wherein the determining step b) further comprises the 

steps of: 

e) determining one or more motion estimations representative of the global 
motion of the frame with respect to one or more respective preceding or succeeding 
other frames (Subramaniyan: [0035]; [0026], motion estimation is computed for blocks 
of image data from a current image frame using one or more previously processed 
image frames ): 

f) determining one or motion estimations respectively representative of the global 
motion of the one or more other frames with respect to the first preceding or succeeding 
frame (Subramaniyan: [0035]; the motion estimation circuit 1 10 can determine the 
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global motion vector by using an average of all final motion vectors in a previous frame ): 
and 

g) accumulating the respective motion estimations to give one or more respective 
overall motion estimations each substantially representative of the global motion of the 
frame with respect to the first preceding or succeeding frame (Subramaniyan: [0035], 
the motion estimation circuit 1 1 0 can determine the global motion vector by using an 
average of all final motion vectors in a previous frame). 

24. Regarding claim 3, Subramaniyan in view of Jinzenji teaches: 

A method according to claim 1 , wherein the selecting step c) further comprises 

testing the motion estimations in turn (Subramaniyan: [0035], comparing the 
difference metric for each of the final motion vectors in the previous frame with a 
predetermined threshold; [0049], This process continues until one of 3 conditions are 
met: [0050], CONDITION 1 : The MSAD is below a threshold THRESH4 , given by: 
THRESH4=A*Q+B); and 

outputting a motion estimation as being representative of the global motion of the 
frame if it passes the test, wherein the test is applied in turn to motion estimations once 
they have been determined (Subramaniyan: [0035], determining the global motion 
vector based on the each of the final motion vectors in a previous frame with a 
difference metric that is below the threshold; [0049], This process continues until one of 
3 conditions are met: [0050], CONDITION 1 : The MSAD is below a threshold 
THRESH4 , given by: THRESH4=A*Q+B), and if the test is passed then no further 
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motion estimations are determined (Subramaniyan: [0049], This process continues until 
one of 3 conditions are met : [0050], CONDITION 1 : The MSAD is below a threshold 
THRESH4, given by: THRESH4=A*Q+B). 

25. Regarding claim 4, Subramaniyan in view of Jinzenji teaches: 
A method according to claim 3, wherein the test comprises 

comparing the motion estimation with a threshold value (Subramaniyan: [0049]- 
[0050], THRESH4), wherein the test is passed if the parameters of the motion 
estimation do not exceed the threshold value (Subramaniyan: [0049]-[0050], [0054] The 
best MV at the end of the second stage is chosen as the best MV for the macroblock; 
the MV having a MSAD below THRESH4 is chosen as the best MV). 

26. Regarding claim 6, Subramaniyan in view of Jinzenji teaches: 

A method according to claim 1 further comprising generating panoramic images 
from a motion-compensated inter-frame encoded video sequence, the generating 
comprising: 

for each frame of the sequence, determining the global motion of each frame with 
respect to the first preceding or succeeding frame using the method of claim 1 (Jinzenji: 
col 8, line 32-56; global motion is obtained in step 1); and 

generating at least one panoramic image representing the frames of the video 
sequence using the frame global motion estimations thus determined (Jinzenji: col 8, 
line 32-56; a provisional sprite <panoramic image> is generated). 
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27. Regarding claim 7, Subramaniyan in view of Jinzenji teaches: 

A method according to claim 6, wherein the generating step further comprises: 

selecting a particular frame of the sequence as a reference frame, the plane of 
the reference frame being a reference plane (Jinzenji: col 8, line 32-56; the reference 
coordinate system which is for the reference frame); 

for each frame other than the reference frame, accumulating the frame global 
motion estimations from each frame back to the reference frame (Jinzenji: col 8, line 32- 
56; each original image of the arbitrary frames is mapped to a reference coordinate 
system which is for the reference frame); 

warping each frame other than the reference frame onto the reference plane 
using the accumulated frame global motion estimations to give one or more pixel values 
for each pixel position in the reference plane (Jinzenji: col 8, line 32-56; frame warping 
occurs when frames are mapped to the reference coordinate system using global 
motion so as to insert or overwrite pixels); and 

for each pixel position in the reference plane, selecting one of the available pixel 
values for use as the pixel value in the panoramic image (Jinzenji: col 8, line 32-56; a 
pixel value of a point is obtained from pixel values which exist in the same point). 

28. Regarding claim 8, Subramaniyan in view of Jinzenji teaches: 

A method according to claim 7, wherein the selecting step comprises selecting a 
substantially median pixel value from the available pixel values for use in a background 
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panoramic image (Jinzenji: col 1 0, line 8-1 1 ; for a plurality of pixels which are mapped 
to the same coordinates, a median value of the pixels is selected as the value of the 
coordinates of the provisional sprite). 

29. Regarding claim 9, Subramaniyan in view of Jinzenji teaches: 

A method according to claim 7, wherein the selecting step comprises selecting a 
substantially most different pixel value from the available pixel values for use in a 
foreground panoramic image (Jinzenji: col 8, line 47-51 ; using a threshold to select the 
most different pixel). 

30. Regarding claim 1 3, Subramaniyan in view of Jinzenji teaches the computer 
readable storage medium storing limitations of this claim as discussed above with 
respect to claim 1 (Subramaniyan: [0085]). 

31. Regarding claim 19, 
Subramaniyan teaches: 

A system for generating panoramic images from a motion-compensated inter- frame 
encoded video sequence, comprising: 

a system for global motion estimation between frames of a motion-compensated 
inter-frame encoded video sequence (Subramaniyan: [0035]; the motion estimation 
circuit 1 1 0 can determine the global motion vector by using an average of all final 
motion vectors in a previous frame ), the system for global motion estimation comprising: 
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video processing means arranged in use to: 

i) determine a motion estimation representative of the global motion 
directly between the particular frame and an preceding or succeeding frame of 
the inter-frame encoded video sequence on the basis of motion vectors 
therebetween (As stated in [0030] of Subramaniyan, the "global motion vector is 
estimated by the motion estimation circuit 110 as the average of all final motion 
vectors of the previous frame". Thus Subramaniyan is able to determine a 
motion estimation representative of the global motion between the particular 
frame and another frame on the basis of an average of motion vectors 
therebetween, which satisfies claim limitation (a). This is reaffirmed in [0035] of 
Subramaniyan, which states that "the motion estimation circuit 110 can 
determine the global motion vector by using an average of all final motion vectors 
in a previous frame".); 

ii) determine one or more further motion estimations representative of the 
global motion between the particular frame and the preceding or succeeding 
frame at least partially on the basis of motion vectors between sub-parts of the 
particular frame and sub- parts of one or more preceding or succeeding other 
frames (As stated in [0034] of Subramaniyan, "further motion vector predictors 
can be determined based on the result of motion estimation done for the same 
macroblock, but on a different previously coded frame". Thus Subramaniyan 
discloses further motion vector predictors based on a previous frame that is 
different from the previous frame initially used. Using these further motion 
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vectors predictors in light of how global motion vectors in Subramaniyan are 
determined, Subramaniyan is capable of determining a further motion estimation 
representative of the global motion between the particular frame and another 
frame at least partially on the basis of an average of motion vectors between the 
particular frame and one or more different other frames, which satisfies limitation 
(b). This is reaffirmed in [0026] of Subramaniyan, which states that "motion 
estimation is computed for blocks of image data from a current image frame 
using one or more previously processed image frames".); and 
a motion estimation selector means arranged in use to select one of the motion 
estimations which meets at least one predetermined criterion as being representative of 
the global motion of the frame (As stated in [0030] of Subramaniyan, the "global motion 
vector is estimated by the motion estimation circuit 1 10 as the average of all final motion 
vectors of the previous frame for which a difference metric was below a certain 
threshold THRESH1 ". Since the final motion vectors are subject to a difference metric 
and the final motion vectors are used to form the global motion vector, the global motion 
vector is subject to a predetermined criterion. By determining global motion in such a 
manner, global motion is contemporaneously selected upon its determination. Thus 
Subramaniyan is capable of selecting one of the motion estimations which meets at 
least one predetermined criterion as being representative of the global motion of the 
frame, which satisfies limitation (c). This is reaffirmed in [0035] of Subramaniyan, which 
states that "the motion estimation circuit 110 can determine the global motion vector by 
calculating a difference metric for each of final motion vectors in a previous frame, 
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comparing the difference metric for each of the final motion vectors in the previous 
frame with a predetermined threshold, and determining the global motion vector based 
on the each of the final motion vectors in a previous frame with a difference metric that 
is below the threshold".); 

and further arranged to provide global motion estimations for each frame 
(Subramaniyan: [0035]; the motion estimation circuit 1 10 can determine the global 
motion vector by using an average of all final motion vectors in a previous frame ); 

Subramaniyan fails to teach: 

panoramic image generating means for generating at least one panoramic image 
representing the frames of the video sequence using the global motion estimations thus 
determined. 

Jinzenji teaches: 

panoramic image generating means for generating at least one panoramic image 
representing the frames of the video sequence using the global motion estimations thus 
determined (Jinzenji col 8, line 32-56; a provisional sprite <panoramic image> is 
generated). 

At the time of invention, it would have been obvious to a person having ordinary 
skill in the art to combine the teachings of Jinzenji with Subramaniyan. Using the 
panoramic generating means of Jinzenji would allow for creation of a panoramic image 
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based on the global motion of Subramaniyan, thereby allowing a user to implement an 
established use for global motion. 

32. Regarding claim 20, Subramaniyan in view of Jinzenji teaches: 
wherein the panoramic image generating means is further arranged in use to: 

select a particular frame of the sequence as a reference frame, the plane of the 
reference frame thereby being a reference plane (Jinzenji: col 8, line 32-56; the 
reference coordinate system which is for the reference frame); 

for each frame other than the reference frame, accumulate the global motion 
estimations from each frame back to the reference frame (Jinzenji: col 8, line 32-56; 
each original image of the arbitrary frames is mapped to a reference coordinate system 
which is for the reference frame); 

warp each frame other than the reference frame onto the reference plane using 
the accumulated global motion estimations to give one or more pixel values for each 
pixel in the reference plane (Jinzenji: col 8, line 32-56; frame warping occurs when 
frames are mapped to the reference coordinate system using global motion so as to 
insert or overwrite pixels); and 

for each pixel position in the reference plane, select one of the available pixel 
values for use as the pixel value in the panoramic image (Jinzenji: col 8, line 32-56; a 
pixel value of a point is obtained from pixel values which exist in the same point). 



33. 



Regarding claim 21 , Subramaniyan in view of Jinzenji teaches: 
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wherein the panoramic image generating means is further arranged to select a 
substantially median pixel value from the available pixel values for use in a background 
panoramic image (Jinzenji: col 10, line 8-11 ; for a plurality of pixels which are mapped 
to the same coordinates, a median value of the pixels is selected as the value of the 
coordinates of the provisional sprite). 

34. Regarding claim 22, Subramaniyan in view of Jinzenji teaches: 

wherein the panoramic image generating means is further arranged to select a 
substantially most different pixel value from the available pixel values for use in a 
foreground panoramic image (Jinzenji: col 8, line 47-51 ; using a threshold to select the 
most different pixel). 

35. Claim 18 is rejected under 35 U.S.C. 103(a) as being unpatentable over 
Subramaniyan et al. (US Patent Application Publication # 2004/0028134 A1) in view of 
Lee et al. (US Patent Application Publication # 2003/0103568 A1). 

36. Regarding claim 18, 
Subramaniyan teaches: 

A system according to claim 16 (as shown above), 



Subramaniyan fails to teach: 
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and further comprising motion estimation interpolation means arranged to 
interpolate between the motion estimations of adjacent frames to give an interpolated 
motion estimation, in dependence on the motion estimation testing means indicating 
that the test has been failed by all the motion estimations determined by the video 
processing means. 

Lee teaches: 

and further comprising motion estimation interpolation means arranged to 
interpolate between the motion estimations of adjacent frames to give an interpolated 
motion estimation, in dependence on the motion estimation testing means indicating 
that the test has been failed by all the motion estimations determined by the video 
processing means (Lee: [0061]-[0063]). 

At the time of invention, it would have been obvious to a person having ordinary 
skill in the art to combine the teachings of Lee with Subramaniyan. The teachings of 
Lee provide for motion compensated interpolation that eliminates blocking artifacts (Lee 
[0048]), thereby increasing the ability of the Subramaniyan teachings to generate 
accurate global motion in the presence of blocking artifacts. 



37. Claim 5 is rejected under 35 U.S.C. 103(a) as being unpatentable over 
Subramaniyan et al. (US Patent Application Publication # 2004/0028134 A1) in view of 
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Jinzenji et al. (US Patent # 6,977,664 B1) and Lee et al. (US Patent Application 
Publication # 2003/01 03568 A1 ). 

38. Regarding claim 5, 

Subramaniyan in view of Jinzenji teaches: 

A method according to claim 3 (as shown above), 

Subramaniyan in view of Jinzenji teaches fails to teach: 
wherein if the test is failed, the method further comprises: 

interpolating between the motion estimations of adjacent frames to give an 
interpolated motion estimation which is then output as the motion estimation 
representative of the global motion of the frame. 

Lee teaches: 

wherein if the test is failed, the method further comprises: 

interpolating between the motion estimations of adjacent frames to give an 
interpolated motion estimation which is then output as the motion estimation 
representative of the global motion of the frame (Lee: [0061]-[0063]). 

At the time of invention, it would have been obvious to a person having ordinary 
skill in the art to combine the teachings of Lee with Subramaniyan in view of Jinzenji. 
The teachings of Lee provide for motion compensated interpolation that eliminates 
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blocking artifacts (Lee [0048]), thereby increasing the ability of the Subramaniyan in 
view of Jinzenji teachings to generate accurate global motion in the presence of 
blocking artifacts. 

39. Claims 1 0-1 1 and 23-24 are rejected under 35 U.S.C. 1 03(a) as being 
unpatentable over Subramaniyan et al. (US Patent Application Publication # 
2004/00281 34 A1 ) in view of Jinzenji et al. (US Patent # 6,977,664 B1 ) and Szeliski et 
al. (US Patent #6,348,91 8 B1). 

40. Regarding claims 10-11, 
Subramaniyan in view of Jinzenji teaches: 

A method according to claim 7 (as shown above), 

Subramaniyan in view of Jinzenji fails to teach: 
wherein the selecting step comprises: 

calculating the mean pixel value of the available pixel values; 

calculating the L1 distance between each available pixel value and the calculated 
mean pixel value; and 

select the pixel value with the median L1 distance for use in a background 
panoramic image. 

select the pixel value with the maximum L1 distance for use in a foreground 
panoramic image. 
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Szeliski teaches: 

wherein the selecting step comprises: 

calculating the mean pixel value of the available pixel values (Szeliski: col 8, line 
57-65; taking the mean of the color or intensity values); 

calculating the L1 distance between each available pixel value and the calculated 
mean pixel value (Szeliski: col 8, line 57-65; where the averaging is weighted by the 
distance of each pixel from the nearest invisible pixel); and 

select the pixel value with the median L1 distance for use in a background 
panoramic image (Szeliski: col 8, line 57-65; using the median technique). 

select the pixel value with the maximum L1 distance for use in a foreground 
panoramic image (Szeliski: col 8, line 57-65; the simplest technique is the median 
technique, but many others exist. This portion of Szeliski discloses blending specifically 
for a background image. This portion of Szeliski also discloses blending generally. 
Instead of using the median technique for blending background pixels, a maximum 
technique is obvious for blending foreground pixels. This is because foreground pixels 
are most different from background pixels). 

It would have been obvious to a person having ordinary skill in the art to combine 
the teachings of Szeliski with Subramaniyan in view of Jinzenji. Using the blending 
technique of Szeliski would smooth out disparities of the panoramic image of 
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Subramaniyan in view of Jinzenji, thus creating a panoramic image with increased 
image quality (Szeliski: col 9, line 6-8). 

41 . Regarding claims 23-24, Subramaniyan in view of Jinzenji teaches the system 
limitations of this claim as discussed above with respect to claims 10-11. 

Conclusion 

42. Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to James Pontius whose telephone number is (571) 270- 
7687. The examiner can normally be reached on Monday - Thursday, 8 AM - 4 PM est.. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Joseph Ustaris can be reached on (571) 272-7383. The fax phone number 
for the organization where this application or proceeding is assigned is 571 -273-8300. 

Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a 
USPTO Customer Service Representative or access to the automated information 
system, call 800-786-91 99 (IN USA OR CANADA) or 571 -272-1 000. 
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